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A method for combining a set of n still image frames into a single reduced noise still im^e The 
invention reduces visable noise, along with some television color artifacts, in still Images captured from a 
video stream by combining data from multiple video frames. Each frame comprises an XY array of pixel data 
(100) In the preferred embodiment, each pixel includes data defining an intensity level as well as color va ues 
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much faster and simpler to implement than true multi^pixel motion estimation, which is not really necessary 
in a series of nominally— still images. 
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METHOD AND SYSTEM FOR COMBINING MULTIPLE IMAGES INTO A 
SINGLE HIGHER-QUALITY IMAGE 

TECHNICAL FIELD 

This invention relates to electronic video capture systems, and more particularly to a 
method and system for combining multiple image frames into a single reduced noise still 
image. 

BACKGROUND 

•"Video still capture" involves a process of extracting a still image from an image set 
comprising a sequence of electronically-encoded image frames. A number of video capture 
products are on the market that allow a user to capture one or more still images from a 
conventional video source to an attached computer, such as a personal computer (PC). The 
video source can be a signal conforming to one of the v^^ell-known NTSC, PAL, or SECAM 
video standards, or a video camera, or sequential images from a digital camera, or recorded 
video signals played on a video cassette recorder (VCR) or the like. 

Many video sources, such as consumer-quality computer cameras and consumer 
VCRs, tend to have noise problems that degrade image quality. When capturing single video 
frames for display as still images, these problems are still visible. Some attempts have been 
made to reduce such noise. For example, the SNAPPY^^* video snapshot product from Play, 
Inc. generates an unv^eighted average of multiple video fields without any motion compensa- 
tion to output a single still image with somewhat reduced noise. 

The inventor has determined that there is still a need for improved noise reduction in 
a video capture system. The present invention provides a method and system for achieving 
this end. 
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SUMMARY 

The invention reduces visible noise, along with some television color artifacts, by 
combining a set of w still image frames into a single reduced noise still image. Each frame 
comprises an XY array of picture element (pixel) data. In the preferred embodiment, each 
pixel includes data defining an intensity level (Y in the YUV color space model) as well as 
color values (UV in the YUV color space model). However, other color space models may be 
used if the models define each pixel as having color space coordinates defining intensity and 
color values, directly or indirectly {i.e., by having separate intensity and color values, or 
where such information can be derived from the color space coordinates, or v»*iere the color 
space coordinates can be separately weighted). A weighted average of several frames of 
video is taken to generate a single still output frame. The weighting is determined adaptively 
in order to reduce inter-frame motion artifricts. 

The resulting ou^ut has substantially less visible noise than a simple unweighted 
average. The algorithm described above is much faster and simpler to implement than true 
multi-pixel motion estimation, which is not really necessary in a series of nominally-still 
unages. If averaging is done on individual NTSC-derived images in the YUV or YIQ color 
spaces, and if weights are based only on pbcel intensity, television-related color artifacts may 
also be reduced by a comb filtering effect. 

The details of one or more embodiments of the invention are set forth in the accompa- 
nying drawings and the description below. Other features, objects, and advantages of the 
invention will be apparent from the description and drawings, and fiom the claims. 

DESCRIPTION OF DRAWINGS 

FIG. 1 is a block diagram of a prior art electronic video tuner system and computer 
system that may be used with the present invention. 

FIG. 2 is a flowchart of a preferred embodiment of the invention. 

Like reference numbers and designations in the various drawings indicate like 
elements. 
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DETAILED DESCRIPTION 

The invention will be described in the context of a conventional image capture 
system the provides a series of decoded video image frames to a buffer memory of a 
computer system. The invention operates on the contents of the buffer memory to provide a 
still image with reduced noise. 

FIG. 1 is a block diagram of a prior art electronic video decoder system 1 and 
computer system 2 that may be used with the present invention. Any of several video 
sources, such as a video camera 3, VCR 4, or a television antenna or cable signal source 5 
processed through an RF tuner 5a, are coupled to the video decoder system 1, which may be, 
for example, a Rockwell Semiconductor Systems Bt848A single-chip video capture proces- 
sor and PCI Bus Master, available from Rockwell Semiconductor Systems, Newport Beach, 
CA. The video decoder system 1 can place video data directly into the system memory 6 of 
the computer system 2 for video capture applications. The computer system 2 includes the 
system memory 6, a central processing unit 7, and a host bridge 8 to couple the computer 
system to a conventional bus 9, such as the PCI Bus. The video decoder system 1 can also 
provide video data to a computer graphics adapter 10, which includes a graphics controller 
1 1 , a frame buffer 1 2, and a RAMDAC 13. The output of the video decoder system 1 is 
placed into a target area in the finme buffer 12 for video overlay applications, so that a video 
image stream may be displayed on a monitor 14. 

The video decoder system 1 includes an analog-to-digital converter (ADC) 20 for 
converting the input video signal to a "raw" digital format. A Y/C separation circuit 21 
separates luminance and color (chroma) information encoded within the digital signal into 
conventional YUV color space data values. The separated infomiation is temporarily stored 
in a first-in-first-out (FIFO) memory 22. A PCI Bus Master circuit 23 provides controlled 
transfer of decoded, digitized video frames over the bus 9 connecting the video decoder 
system 1 to the computer system 2. (Although the illustrated embodiment shows use of a PCI 
bus, the invention is not limited to any particular bus). A video synchronization and scaling 
circuit 24 locks the video decoder system 1 to the incoming analog video signal and allows 
resizing of the video image size. Control of the video decoder system 1 is provided through 
either a general purpose input/output port 25 or a two-wire inter-integrated circuit interface 
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26. In an alternative embodiment, Y/C separation may be performed within the computer 
system 2 using appropriate progranmiing. 

For purposes of the invention, the video decoder system 1 is used to place data for a 
series ofstiU video fiames 6-1 ... 6-« directly into the system memory 6 of the computer 
system 2, as shown in HG. 1 . If the data is not in YUV format, Y/C separation is performed 
by appropriate processing of such data to generate YUV values for each pixel of the series of 
still video frames 6-1 ... 6-n. One method of separating luminance (Y) from chrominance (C) 
in a video signal is simply to use a notch filter to generate Y, and a bandpass filter to extract 
C. The U and V components of the YUV color space are derived from C by quadrature 
amplitude demodulation, in known fashion. Thereafter, in the preferred embodiment, a 
computer implementation of the method described below is applied to a set of n video frames 
to derive a still image with reduced visible noise. The computer implementation preferably 
executes on the CPU 7 of the computer system 2. 

The invention reduces visible noise, along with some television color artifects, in still 
images captured from a video stream by combining data from multiple video fiames. Each 
frame comprises an XY array of picture element (pixel) data. In the preferred embodiment, 
each pixel includes data defining an intensity level (Y m the YUV color space model) as well 
as color values (UV in the YUV color space model). A weighted average of several frames of 
video is taken to generate a single still output frame. The weighting is determined adaptively 
in order to reduce inter-frame motion artifects. 

More particularly, one frame near the middle (in time) of a sequence of « frames is 
designated the "primary" frame P. Its weight, JVp, is "1" for every pixel. The intensity level 
Yp of each pixel of the primary frame is compared to the intensity level of the corresponding 
pixel of each of the other «-l bracketing frames. Based on the absolute difference between 
the Y value of a primary-frame pixel and that of another frame «, a weight Wn is generated. 
Thus, every pixel location has its own set of n weights. Each weight Wn is set to one if the 
difference is below a selected threshold Tl and to zero if the difference is above a selected 
threshold T2. The values for Tl and T2 may be empiricaUy determined. Between the two 
thresholds, the weight varies from one to zero, preferably linearly (although other fimctions 
may be used). To summarize, each weight Wis calculated as follows: 
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If|Xp-Y«|<Tl,FF/i=l 
Else If |Y;7 - Yn| > T2, ff>2 = 0 
Else Wn = (|Y;7 - Y«| - T2) / (T1-T2) 

5 Once weights have been determined, YUV values for each output pixel of the desired 

still frame are calculated as follows: 

Yout = (FT,* Y, + . . . + W*Y„) / (FF, + , . . + fTJ 
Uout = (frt*U, + ... + W*\J„) /(W,-^ .,. + W„) 
Vout = iJV*V, + ... + W*W„) / + ... + »;) 
10 where (Y„ U„ V,) . . . (Y„, U„, V J are the corresponding pixel values from each frame 

1 to n. 

The resulting output has substantially less visible noise than a simple unweighted 
average. The algorithm described above is much faster and simpler to implement than true 
15 multi-pixel motion estimation, which is not really necessary in a series of nominally-still 

image. Since averaging is done in the YUV color space, and since weights are based only on 
intensity rather than partially on color information, television-related color artifacts are 
reduced. 

While the YUV color space model is shown in the illustrated embodiment, other 
20 color space models may be used if the models define each pixel as having color space 

coordinates defining intensity and color values, directly or indirectly. For example, a suitable 
color space model may have separate intensity and color values, or such information can be 
derived from the color space coordinates, or the color space coordinates can be separately 
weighted. For instance, for an RGB color space model, there is no separate intensity channel. 
25 However, each color axis (R, G, B) can be weighted separately, or the RGB values for a pixel 
can be used to calculate back to an intensity value which is used for weighting in accordance 
with the invention. Other color space models to which the invention can be adapted are 
YCrCbandYIQ. 

30 FIG. 2 is a flowchart of a preferred embodiment of the invention. Initially, a set of 

still frames are generated by the video decoder system 1 and placed in the system memory 6 
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of the computer system 2 (STEP 100). Each ftame comprises X by Y pixels, each pixel 
having color space coordinates defining inteiisity and color values, directly or indirectly 
ie.g., YUV) values. Next, for each pixel in a primary fiame P, a set of « intensity-based 
weights relative to corresponding pixels in n-l bracketing flames are computed as described 
5 above (STEP 1 02). Color and intensity values are computed for each pixel of an output frame 
as the weighted average of each corresponding pixel of the n frames, as described above 
(STEP 104). Lastly, the computed color and intensity values of the output frame are output 
(e.g., to a monitor or data file) as a reduced noise still frame (STEP 106). 

10 Computer Implementation 

Aspects of the invention may be implemented in hardware or software, or a combina- 
tion of both. However, preferably, the algorithms and processes of the invention are imple- 
mented in one or more computer, programs executing on programmable computers each 
comprising at least one processor, at least one data storage system (including volatile and 

15 non-volatile memory and/or storage elements), at least one input device, and at least one 
output device. Program code is applied to input data to perform the ftmctions described 
herein and generate output information. The output information is applied to one or more 
ou^ut devices, in known frisfaion.. 

Each program may be implemented in any desued computer language (including 

20 machine, assembly, high level procedural, or object oriented programming languages) to 
communicate witli a computer system, hi any case, the language may be a compiled or 
interpreted language. 

Each such computer program is preferably stored on a storage media or device (e.g., 
ROM, CD-ROM, tape, or magnetic diskette) readable by a general or special purpose 

25 programmable computer, for configuring and operating the computer when the storage media 
or device is read by the computer to perform the procedures described herein. The inventive 
system may also be considered to be implemented as a computer-readable storage mediimi, 
configured with a computer program, where the storage medium so configured causes a 
computer to operate m a specific and predefined manner to perform the fimctions described 

30 herein. 
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A number of embodiments of the present invention have been described. Neverthe- 
less, it will be understood that various modifications may be made without departing firom 
the spirit and scope of the invention. For example, the captured still frame may be directly 
output to a printer, or transmitted electronically to another computer. Accordingly, other 
embodiments are within the scope of the following claims. 
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WHAT IS CLAIMED IS: 



1 LA method for combining a set of n still image frames into a single reduced noise still 

2 image, each frame comprising X by Y pixels, each pixel having color space coordinates 

3 defining intensity and color values, including the steps of: 

4 (a) for each pixel in a primaiy frame P of the set of still image frames, computing a set 

5 of « intensity-based weights Ww relative to corresponding pixels in n-\ bracketing 

6 still image frames; 

7 (b) computing color and intensity values for each pixel of an output frame as the 
B weighted average of each corresponding pixel of the n still image frames; 

9 (c) outputting the computed output frame as a reduced noise still frame. 

1 2. The method of claim 1 , wherein each pixel in each still image frame n has an intensity 

2 value Yw, wherein each pixel in the primary frame P has an intensity value Yp, and the 

3 step of computing a set of n intensity-based weights Wn includes the step of computing 

4 each weight W« according to the relationships: 

5 lf\Yp'Yn\<Tl,Wn^l 

6 Else If I Yj7 . Yn\ > T2, Wn=-0 

7 Else Wn = QYp - Yn\ - T2) / (T1-T2), 

8 where Tl and T2 are selectable thresholds. 
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3 . The method of claim 2, wherein each pixel in each still image fiame n has color values 
Un and Yn, and wherein the step of computing color and intensity values for each pixel 
of an output frame includes the step of computing YUV values for each pixel of the 
output frame according to the relationships: 

Yout = (W*Y, + ... + JV*Y„) / (PFj -f ... -f W„) 

Uout = (J^,*Uj + ... + W*l]„)/(W, + ... + W„) 

Vout = (Wri*V, + ... + W*V„)/{W, + ... + w„x 
where (Y^, U,, V,) . . . (Y„, V„) are the corresponding pixel intensity and color values 
from each still image frame 1 to n. 
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4. A system for combining a sequence of electronically-encoded image frames into a single 
reduced noise still image, including: 

(a) a video decoder system and computer system for generating a set of n still image 
frames, each still image frame comprising X by Y pixels, each pixel having color 
space coordinates defining intensity and color values; and 

(b) a progranmied processor in the computer system for: 

(1) computing a set of n intensit>'-based weights Ww relative to corresponding 
pixels in n-1 bracketing still image frames for each pixel in a primary frame P 
of the set of still frames; and 

(2) computing color and intensity values for each pixel of an output frame as the 
weighted average of each corresponding pixel of the n still image frames, the 
output frame having reduced noise. 

5. The system of claim 4, fiuther including an output device, coupled to the computer 
system, for outputting the computed output frame as a reduced noise still frame. 

6. The system of claim 4, wherein each pixel in each still image frame n has an intensity 
value Y«, wherein each pixel in the primary frame P has an intensity value Yp, and the 
program for computing a set ofn intensity-based weights Ww ftirther computes each 
weight Ww according to the relationships: 

If|Yp.Y«|<Tl,fF«=l 
Else If |Yp - Yn\ > T2, Wn^O 
Else Wn = QYp - Yn\ - T2) / (T1-T2), 
where Tl and T2 are selectable thresholds. 
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7. The system of claim 6, wherein each pixel in each still image frame n has color values 
U» and V«, and wherein the program for computing color and intensity values for each 
pixel of an output firame further computes YUV values for each pixel of the output frame 
according to the relationships: 

Yout = (JT,* Yj + . . . + W*YJ / (fF, + . . . + PTJ 

Uout = (fF,*U, + ... + W„^U,)/(W, + ... + W„) 

Vout = (Pr,*V, + ... + PF„*V„)/(H>^j + ...4-)Pr„), ' 
where (Y„ U„ V,) . . . (Y„, V„) are the corresponding pixel intensity and color values 
from each still image frame 1 to w. 
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8. A computer program, residing on a computer-readable medium, for combining a set ofn 
still image frames into a single reduced noise still image, each frame comprising X by Y 
pixels, each pixel having color space coordinates defining intensity and color values, the 
computer program comprising instructions for causing a computer to: 

(a) for each pixel in a primary frame P of the set of still image frames, compute a set of 
n intensity-based weights Ww relative to corresponding pixels in nA bracketing still 
image frames; 

(b) compute color and intensity values for each pixel of an output frame as the weighted 
average of each corresponding pixel of the n still image frames; 

(c) output the computed output frame as a reduced noise still frame, 

9. The computer program of claim 8, wherein each pixel in each still image frame n has an 
intensity value Y«, wherein each pixel in the primary frame P has an intensity value Yp, 
and the instructions for causing a computer to compute a set of n intensity-based weights 
W« further includes instructions for causing a computer to compute each weight Ww 
according to the relationships: 

lf\Yp-Yn\<TUWn=l 
Else If lYp - Y«| > T2, Wn = 0 
Else JVn = QYp - Yn\ - T2) / (T1-T2), 
where Tl and T2 are selectable thresholds. 
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10. The computer program of claim 9, wherein each pixel in each still image frame n has 
color values Un and Yn, and wherein the instmctions for causing a computer to compute 
color and intensity values for each pixel of an output frame further includes instmctions 
for causing a computer to compute YUV values for each pixel of the output frame 
according to the relationships: 

Yout = {W*Y, + . . . + W*Y„) /(W,-^ .,. + W„) 

Uout = (W,*U, + ... W*V„) / (JT, + ... + r„) 

Vout = (PF,*V, + . . . + W„*W„j / (FT, 4- ... + 
where (Yj, U„ V,) . . , (Y„, U„, W„) are the corresponding pixel intensity and color values 
from each still image frame 1 to w. 
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